3 research outputs found
Enhancing Traffic Prediction with Learnable Filter Module
Modeling future traffic conditions often relies heavily on complex
spatial-temporal neural networks to capture spatial and temporal correlations,
which can overlook the inherent noise in the data. This noise, often
manifesting as unexpected short-term peaks or drops in traffic observation, is
typically caused by traffic accidents or inherent sensor vibration. In
practice, such noise can be challenging to model due to its stochastic nature
and can lead to overfitting risks if a neural network is designed to learn this
behavior. To address this issue, we propose a learnable filter module to filter
out noise in traffic data adaptively. This module leverages the Fourier
transform to convert the data to the frequency domain, where noise is filtered
based on its pattern. The denoised data is then recovered to the time domain
using the inverse Fourier transform. Our approach focuses on enhancing the
quality of the input data for traffic prediction models, which is a critical
yet often overlooked aspect in the field. We demonstrate that the proposed
module is lightweight, easy to integrate with existing models, and can
significantly improve traffic prediction performance. Furthermore, we validate
our approach with extensive experimental results on real-world datasets,
showing that it effectively mitigates noise and enhances prediction accuracy
DiffTraj: Generating GPS Trajectory with Diffusion Probabilistic Model
Pervasive integration of GPS-enabled devices and data acquisition
technologies has led to an exponential increase in GPS trajectory data,
fostering advancements in spatial-temporal data mining research. Nonetheless,
GPS trajectories contain personal geolocation information, rendering serious
privacy concerns when working with raw data. A promising approach to address
this issue is trajectory generation, which involves replacing original data
with generated, privacy-free alternatives. Despite the potential of trajectory
generation, the complex nature of human behavior and its inherent stochastic
characteristics pose challenges in generating high-quality trajectories. In
this work, we propose a spatial-temporal diffusion probabilistic model for
trajectory generation (DiffTraj). This model effectively combines the
generative abilities of diffusion models with the spatial-temporal features
derived from real trajectories. The core idea is to reconstruct and synthesize
geographic trajectories from white noise through a reverse trajectory denoising
process. Furthermore, we propose a Trajectory UNet (Traj-UNet) deep neural
network to embed conditional information and accurately estimate noise levels
during the reverse process. Experiments on two real-world datasets show that
DiffTraj can be intuitively applied to generate high-fidelity trajectories
while retaining the original distributions. Moreover, the generated results can
support downstream trajectory analysis tasks and significantly outperform other
methods in terms of geo-distribution evaluations
MOELoRA: An MOE-based Parameter Efficient Fine-Tuning Method for Multi-task Medical Applications
The recent surge in the field of Large Language Models (LLMs) has gained
significant attention in numerous domains. In order to tailor an LLM to a
specific domain such as a web-based healthcare system, fine-tuning with domain
knowledge is necessary. However, two issues arise during fine-tuning LLMs for
medical applications. The first is the problem of task variety, where there are
numerous distinct tasks in real-world medical scenarios. This diversity often
results in suboptimal fine-tuning due to data imbalance and seesawing problems.
Additionally, the high cost of fine-tuning can be prohibitive, impeding the
application of LLMs. The large number of parameters in LLMs results in enormous
time and computational consumption during fine-tuning, which is difficult to
justify. To address these two issues simultaneously, we propose a novel
parameter-efficient fine-tuning framework for multi-task medical applications
called MOELoRA. The framework aims to capitalize on the benefits of both MOE
for multi-task learning and LoRA for parameter-efficient fine-tuning. To unify
MOE and LoRA, we devise multiple experts as the trainable parameters, where
each expert consists of a pair of low-rank matrices to maintain a small number
of trainable parameters. Additionally, we propose a task-motivated gate
function for all MOELoRA layers that can regulate the contributions of each
expert and generate distinct parameters for various tasks. To validate the
effectiveness and practicality of the proposed method, we conducted
comprehensive experiments on a public multi-task Chinese medical dataset. The
experimental results demonstrate that MOELoRA outperforms existing
parameter-efficient fine-tuning methods. The implementation is available online
for convenient reproduction of our experiments